Global-Reasoned Multi-Task Learning Model for Surgical Scene Understanding

Global and local relational reasoning enable scene understanding models to perform human-like analysis understanding. Scene enables better semantic segmentation object-to-object interaction detection. In the medical domain, a robust surgical model allows automation of skill evaluation, real-time monitoring surgeon’s performance post-surgical analysis. This letter introduces globally-reasoned multi-task capable performing instrument tool-tissue Here, we incorporate global in latent space introduce multi-scale (neighborhood) coordinate improve segmentation. Utilizing setup, visual-semantic graph attention network detection is further enhanced through reasoning. The features from module are introduced into network, allowing it detect interactions based on both node-to-node Our reduces computation cost compared running two independent single-task by sharing common modules, which indispensable for practical applications. Using sequential optimization technique, proposed outperforms other state-of-the-art MICCAI endoscopic vision challenge 2018 dataset. Additionally, also observe when trained using knowledge distillation technique. official code implementation made available GitHub.